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SIMULATION FIDELITY AND NUMEROSITY 
EFFECTS IN CDTI EXPERIMENTATION 

The NASA- Ames agreement No. NCC 2-93 between NASA-ARC and Tufts University 
in effect from September 1980 to August 1981 and by extensions to December 1982 had 
as its major thrust veridical tests of Simulator Numerosity and Fidelity as affecting 
CDTI experimental results. This will be briefly commented on as several papers de- 
scribing results have been presented in conference. 

A. VERIDICAL TESTS OF SIMULATOR NUMEROSITY AND FIDELITY. 

A series of protocols had been devised and discussed with ARC personnel as tests 
for determining the differences in experimental results attributable to the levels of 
numerosity and fidelity of CDTI simulators in a CDTI scenario. 

Although extensive work had been completed on using the Boston-Logan area as the 
traffic setting, work based on interviews with pilots and controllers and tower per- 
sonnel, an existing traffic configuration based on the San Jose area was used instead. 

A series of 4 experimental comparisons were run at ARC during August 1981 test- 
ing various levels of simulator fidelity and CDTI numerosity. The conditions were: 

H7 : 3 simulators with CDTI + 4 pseudopilots with CDTI 
H2 » 2 simulators with CDTI + 1 simulator without CDTI and 
4 pseudopilots without CDTI 

HO : 3 simulators without CDTI + 4 pseudopilots without CDTI 
L2 ; 2 simulators with CDTI + 5 keyboard targets without CDTI. 

Each of the 4 conditions were run twice per group with 3 groups. Each group included 
an air traffic controller in addition to the required number of pilots (7). 

The scenario required A/C scheduled into the San Jose area to be sequenced, merged 
and landed. At some point in the ongoing task, an emergency was declared and the 
runway was closed for approximately 5 minutes. A complete run including the emer- 
gency lasted 30 minutes. During each run complete data were taken on the individual 
A/C. In addition, all verbal messages were tape-recorded and post flight question- 
naires were filled out by the participants. 

The results of the experiment are extensive and a number have been reported pre- 
viously , These are summarized here with comparison to previously acquired 

CDTI simulation-experimental ones. A full description of the experimental set up, 
etc. can be found in (Chappell k Kreifeldt, 1982). 
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B. SIMULATOR FIDELITY 

There were three levels of simulator fidelity, the"cabs H (air carriers) which were 
the most realistic as to control inputs, dynamics, etc.; "pseudos" (general aviation) 
which were joystick/throttle table top controls* and the leapt realistic "keyboard" 
which permitted one pilot to command multiple targets through simple keyboard inputs. 
There were a number of statistical differences attributed to the simulator fidelity. 

The general aviation targets were issued more speed, heading and altitude changes 
when these were individually piloted as "pseudos" than when they were all under control 
of a single pilot (keyboard). This, of course, infers that communications per target 
would be greater for the more faithful simulation, a fact supported by an analysis of 
the verbal data showing that the GA targets received about 10% more communications 
as "pseudos" than as "keyboards ". Further and although confounded with target type 
(air carrier -most realistic or general aviation - less realistic), the cabs (air carrier) 
received about 40% more communications than the less realistic GA simulators. 

Thus, the controller's verbal workload* a primary measure of workload and capacity 
could be seriously underestimated with .lower fidelity target. This could have potential- 
ly serious consequences on the safety of a system as estimated by using low fidelity 
simulators when verbal workload is taken as a safety measure. 

Compared to the air carrier targets (most realistic simulators), the GA targets had 
the greater altitude and localized errors at the OM and the greater horizontal path 
length. The errors were probably greater because the GA simulator instrumentation 
was quite primitive compared to that of the air carriers. The longer G A horizontal 
path length was most likely attributable to the ATC prediliction for giving air carriers 
preference over GA. 

The intercrossing times at the OM and MM were greatest for the keyboard targets 
and least for the air carriers with somewhat more tighter controlled (smaller variance) 
as well. The better instrumentation in the air carriers compared to the lower fidelity 
may account for their better delivery to the OM. The keyboard targets had the longest 
intercrossing times at both MM and OM. Thus if a low fidelity (keyboard) target facil- 
ity such as at NAFEC is used to generate controllable targets for ATC capacity simula- 
tion studies, the capacity may again be seriously underestimated. 

Two air carriers were within 1 nm horizontal separation longer than any of the other 
4 pairing of air carrier, pseudos and keyboard targets (pseudos and keyboard targets 
were never used at the same time) while two pseudos were within 2. 5 nm and 3. 0 nm 
horizontal separation longer than any of the other 4 pairings. 
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C. FACILITY SIMULATION LEVEL 

There were two facility fidelity levels; a HIGH fidelity level of 3 aircarriers (cabs) 
and 4 GA pseudos or a LOW fidelity level of 2 air carriers and 5 GA keyboard targets. 
In either case there were 7 controllable targets in the problem at any time. The fol- 
lowing statistical differences were attributed to the facility simulation level. 

The GA targets received more altitude, heading and speed changes in the HIGH than 
in the LOW facility fidelity level which reiterates the previously stated results for in- 
dividually piloted GA targets (pseudos) as opposed to the multiply commanded GA tar - 
gets (keyboard). The LOW fidelity facility produced about 30% longer intercrossing 
times at the MM and OM than did the HIGH fidelity level. As the previous section 
indicated, this was attributable to the longer times with keyboard and GA targets than 
with the individually piloted GA targets (pseudos). Thus the lower the overall fidelity 
level of a facility the more seriously the runway capacity may be underestimated. 

D. CDTI EFFECTS 

Although the experiment was not strictly a CDTI one, there were several statistical 
differences attributable to CDTI/nonCDTI conditions without, however, distinguishing 
separately the simulator fidelity or facility fidelity variables. 

Perhaps the most relevant finding is that as the number of CDTI equipped targets 
increased from zero to two (air carriers) to seven (3 air carriers and 4 pseudos), the 
number of communications per target increased as well. There was a 18% increase 
with two CDTI’s compared to none and a 23% increase with seven compared to none. 
Thus there was a slight (4%) increase with 7 CDTI’s compared to 2. This suggests 
that the controller verbal workload would be higher with CDTI. However, the com- 
munication with CDTI’s must be of the advisory type. GA targets with CDTI made 
70% more speed changes, 34% more altitude changes and had a 12% shorter horizon- 
tal path length with CDTI than without. The CDTI equipped targets also had 27% 
more communications than the nonCDTI equipped ones. Again, these communications 
must be of the advisory nature. A surprising finding was that the intercrossing times 

variance was about 8% greater for CDTI equipped targets. This is counter to a pre- 
(14) 

vious study 1 which showed that CDTI equipped simulator air carrier targets had 
a smaller intercrossing time variance than did nonCDTI targets. 

There was no statistical difference between CDTI/nonCDTI targets in terms of 
spacing violations. Thus many CDTI's were no more unsafe than fewer or no CDTI 
targets . 
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E. DISCUSSION 

The experiment has demonstrated that in fact results for various system measures 
important in ATC and CDTI studies may depend upon the level of the facility fidelity 
and the number of actual individually piloted CDTI targets used to simulate a multi- 
CDTI traffic environment. The magnitude of the difference will, of course, depend 
on the scenarios chosen. In particular, the use of a cluster of keyboard controlled 
targets commanded by one ’‘pilot”, a common way of simulating controllable traffic 
in a simple, economic fashion, produces results on a number of measures consider- 
ably different from those obtained using a one target-one pilot realization. The ATC 
method could be expected to differ in significant ways in a one target-one pilot (real- 
istic) environment compared to a multiple target -one pilot one. This difference would 
thus cause the results obtained from a low fidelity, few CDTI simulations to be strong- 
ly suspect when attempting to transfer those results to a real world multi CDTI en- 
vironment. While the experimental scenario used produced statistically significant 
differences ranging from a few percent to 70%, this one experiment can only demon- 
strate the fact that simulator fidelity and CDTI numerosity do effect both magnitude 
and direction of results. By itself it cannot set absolute correction factors to apply 
to the measures for different levels of fidelity and numerosity for any arbitrary 
scenario. However, in as much as the scenario and protocol of the experiment were 
quite realistic and open, the validity of the comparison results seems assured. 

Other experiments using other scenarios would have to be run to establish solid cor- 
rection factors for measures dependent on fidelity and numerosity. 

Over the years since ATC-CDTI studies were begun at NASA-ARC under a NRC 
fellowship in 1972, the experimental scenarios have become more realistic in being 
less controlled and better simulated increasing at least the face validity of the results. 
As control over the experiment is loosened to permit more realistic behavior, the 
need for multivariate statistical methods of analysis and description increases. 

It is strongly suggested that these methods which include discriminant analysis, clus- 
tering, hierarchical techniques, multidimensional scaling, etc., be studied and applied. 
An example of the use of these techniques can be found in Kreifeldt, Parkin and Hart 
(1977) and Kreifeldt and Parkin (1975). 

F. PREVIOUS CDTI RESEARCH UNDER NASA-ARC FUNDING 

The author began ATC-CDTI studies at NASA-ARC in 1973 under a NRC Fellowship 
and subsequently under a series c£ consortium fundings and grants. The major experi- 
ments are summarized in Kreifeldt (1980). Considerable prototyping work was also 
conducted at Tufts University on development of a minicomputer based low cost, multi- 
simulator facility the outlines of which are summarized in Kreifeldt (1981). 
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A full listing of all the papers, presentations and reports produced under the grants 
on the grant matters are given in the Bibliography. 


MINICOMPUTER BASED MULTIMAN SIMULATOR FACILITY 

In addition to the above work, the contents of the minicomputer multicab prototyping 
facility have been shipped to Dr. Diane Damos at Arizona State University on October 
1982, This included two desk top simulators, two computer systems, specially de- 
veloped hardware /software and various manuals and drawings. The work at Tufts on 
this prototype facility as presented in various papers and reports had produced a pro- 
totype facility that could support simultaneously up to eight minicomputer based flight 
simulators of fairly high fidelity. The advantage of the facility would be its flexibility, 
low cost and reliability as well as its ability to stand alone for many types of ex- 
periments . 

As no further development support was obtained for this facility under the NCC 2-93 
agreement, progress stopped in September of 1980. However, a brief progress re- 
port by Mr. Gary MacDonald was prepared on September 2, 1980. This report is 
enclosed and supplements the series of technical reports and papers given earlier and 
listed in the Bibliography. 

This bibliography lists various papers and reports describing work done under 
NASA grant NSG 2056 to prototype a minicomputer based simulation facility at Tufts. 


Kreifeldt, J. G., Gallagher, O. , "Design Outline for a New Multiman ATC Simulation 
Facility at NASA- Ames Research Center, " Proc. 13th Annual Conference on 
Manual Control, MIT, Cambridge, Massachusetts June 1977. 

Kreifeldt, J.G. , ’^Progress on the Multiman ATC Simulation Facility at NASA- Ames 
Research Center, " Proc. 14th Annual Conference on Manual Control, Los Angeles, 
April 1978. 

Kreifeldt, J.G. , "Low Cost Programmable Multisimulator Facility, " Proc. 4th 

AIAA/IEEE Digital Avionics Systems Conference, St. Louis, MO. Nov. 17-19, 1981, 

Kreifeldt, J.G. , "A Study of Pilot-Controller Interaction Under Various Simulated 
Advanced AT A and Navigation Concepts." Progress Report 6/1/77-5/1/78, 
to NASA-ARC, May 1978. 

Kreifeldt, J.G. , "A Study of Pilot-Controller Interaction Under Various Simulated 
Advanced ATC and Navigation Concepts," Progress Report 5/1/78 to NASA-ARC, 
December 1978. 

Kreifeldt, J.G. , "A Study of Pilot -Controller Interaction Under Various Simulated 
Advanced ATC and Navigation Functions, " Final Progress Report 12/1/78- 
8/31/79, Sept. 1979, NASA: NSG-2 156. 

MacDonald G., "Multiman Flight Simulator System." Progress report for the 
period Oct. 1979-Sept. 1980, Tufts University, Department of Engineering 
Design, Sept. 2, 1980. 
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